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(57) Abstract 

Our method and system for enumerating 
a minimal number of test cases for systems 
with interacting elements operates relationships 
between the elements and the number of char- 
acteristics evaluated for each element. In our 
inventive method as executed in a computer 
system (16), the user enters values for each 
of the elements and then defines relationships 
(18) between the elements. Our method then 
enumerates a table of test cases (14) for each 
relationship between elements using determin- 
istic procedures, when applicable, and random 
procedures when the deterministic procedures 
are not applicable. After a table of test cases 
is generated for each of the relation, our inven- 
tive method combines (20) relationships into a 
single table of test cases. 
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Method and System for Automatically 
Generating Efficient Test Cases for Systems 
Having Interacting Elements 

TECHNICAL F IELD OF THE INVENTION 

This invention relates to the testing of computer 
software. More specifically, the present invention relates to 
systems and processes for the efficient generation and 
enumeration of a minimal number of test cases necessary to 
robustly test a software system. 

BACKGROUND OF THE INVENTION 

Transactional computer systems are becoming larger and more 
complex. Telephone companies have implemented transaction 
processing to improve efficiencies in their operational support 
systems. Typically, users interact with these transaction 
processing systems by entering data in fields on a CRT screen and 
then expecting a response in other fields on the screen. As new 
software is developed, these software systems have to be tested 
before they are delivered to end users. 

Software testers test by developing test cases that 
specify combinations of screen inputs which are entered into 
the system to see if the response of the system is appropriate 
as a result of the inputs. One method of testing would be to 
create test cases for all combinations of all input values on 
the screen. For most systems, because of the large number of 
screens with multiple fields each with multiple values, the 
number of test cases that can be generated is extremely large. 
Accordingly, the cost of conducting such tests could be 
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exorbitant. Therefore, in the current art the number of test 
cases is reduced by taking sample values for each field; such as 
the minimum value, a maximum value, and some intermediate value 
for a numeric field. Even so, the cost of conducting screen 
5 testing in the current art can increase the cost of software 
development by up to 30%. These problems are not unique to 
screen based software systems. In the area of protocol 
conformance testing, protocols and features have typically been 
modeled as finite state machines. However, the number of test 

10 cases to test a finite state machine with n states is a 

polynomial in n. Since protocols typically have thousands of 
states, and may have millions, traditional finite state machine 
based testing produces an unreasonable number of tests. The 
example that follows highlights some of the problems. 

15 Assume a system with one screen and n fields on this 

screen. Suppose also that after an analysis of the system 
requirements used to create the software, only a limited number 
of inputs (i) is considered for testing. Then the number of 
possible test cases, each containing values for all fields on 

2 0 the screen is i n . This can be quite large even for a single 

screen. As an example, suppose only three values (i=3) are to be 
considered for each field and there are 13 fields (n=13) on the 
screen then the number of exhaustive test cases are 
3 13 =1, 594, 323 . Obviously, this number is very large and 

25 exhaustive testing is far beyond anyone's available resources. 
Thus, typical testers find various ways of cutting down the 
number of combinations, e.g. by gut feel, or by just trying out 
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all the values one field at a time, holding all other fields at 
default values, unfortunately, this method of default testing 
will not take into account interactions, or dependencies 
between various fields. 

Some researchers in the field have examined the 
application of experimental design techniques used for planning 
experiments to the area of software testing (Phadke, M.S., 
"Quality Engineering Using Robust Design", Prentice Hall, 
Englewood Cliffs, NJ., 1989). Specifically, Pahdke proposes the 
use of orthogonal array designs for testing software. 
Orthogonal array designs are test sets such that tor any pair of 
fields all combinations of input levels occur exactly an equal 
number of times. As an example, consider the situation where one 
screen has three fields (n=3) with two possible inputs per field 
(i=2). An exhaustive test of all possible combinations would 
result in 8 test sets as shown in Table 1 below. 

Table 1: Exhaustive Test Array 
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Using experimental design techniques, a corresponding 
orthogonal array of the same fields and values results in 4 test 
cases as shown in Table 2 below. 



Table 2: Orthogonal Test Array 
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Table 2 depicts every pair of inputs exactly once and has only 
four test cases. This is a 50% reduction in test cases from the 
fully enumerated set of test cases shown in Table 1. 

While orthogonal arrays provide test sets that cover 
every pair of inputs with many fewer test cases than the 
exhaustive test set, this prior art solution has a number of 
problems. First and foremost, not every combination of fields 
and inputs has a corresponding orthogonal array. For example, 
when there are 7 fields and each field has 6 possible inputs, an 
orthogonal array does not exist. Another problem with the 
orthogonal array is that they are difficult to construct; there 
is no single unified way of generating an orthogonal array. This 
difficulty is exacerbated when there are an unequal number of 
values for each of the fields. Orthogonal arrays are also 
wasteful in that they require each pair of inputs to occur 
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exactly the same number of times. This property of balance is 
often necessary in statistical experiments where equal 
precision is required. However, in software testing it is 
unnecessary to require an equal number of replicates. 
5 Therefore, an object of our invention is an automated 

method and system that overcome the limitations in the prior art 
and incorporate interactions between fields while reducing the 
number of test cases necessary to robustly test software 
systems . 

10 SUMMARY OF THE INVENTION 

Our invention is a method and system for enumerating a 
minimal number of test cases for systems having interacting 
elements. As described hereing for screen based software 
systems, our invention operates on a construct of fields, the 

15 number of field values, and relationships between fields. In 
our method, as executed in a computer system, the user enters 
values for each of the fields, defines relationships between 
the fields, and then defines the degree of interaction between 
fields. Our method then enumerates a table of test cases for 

20 each relationship between fields using deterministic 

procedures, when applicable, and random procedures when the 
deterministic procedures are not applicable. After a table of 
test cases is generated for each relationship, our inventive 
method combines the test cases for different relationships into 

25 a single table of test cases. In other applications where a 
system is comprised of component elements with different 
characteristics, our invention treats component elements as 
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analogous to fields and component characteristics as analogous 
to field values. 

BRIEF DESCRIPTION OF THF. DRAWINGS 

Figure 1 illustrates our inventive system. 

Figure 2 depicts a sample input screen of an embodiment of 
our sample system. 

Figure 3 depicts as an example of a relation and a table 
of test cases generated using our projective plane algorithm. 

Figure 4 depicts a table of test cases created using our 
product rule procedures . 

Figure 5 depicts two tables created in accordance with 
our reduced product rule procedures. 

Figure 6 depicts tables of test cases for three types of 
relations . 

Figure 7 depicts a consolidated table of test cases 
created using our test case merging process. 

DETAILED DESCRIPTION 

Our invention is a method and system for enumerating a 
minimal number of test cases that incorporates the interactions 
between fields and which overcomes the limitations of the prior 
art. 

An illustrative embodiment of our invention is shown in 
Figure 1 which includes a screen display device 10 on which the 
screen shown in Figure 2 would be displayed. Users would enter 
values for fields on the screen 28 (or over a plurality of 
screens) and define the relationships between these fields 
(hereinafter called a relation) which establish validation 
rules for groups of the fields. The user would also enter 
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constraints and a parameter specifying the degree of field 
interaction 29 for each relation. Users may also specify the 
system's correct response to the input combinations generated 
from a relation. For example, the proper system response to an 
5 input may simply be a message that the inputs are legal or the 
proper response may be an action such as setting up a telephone 
call. The display controller process 12 would store these field 
relations and validation rules in the relation database 18. 
Tables of test cases for each relation are created in process 14 

10 and then stored also in database 18. After all the data is 

entered, the field relationships defined, and individual tables 
of test cases generated, the user would request the generation 
of a consolidated table of test cases.. The test script merger 
process 20 would then generate a single table of test cases 

15 combining the tables of test cases generated by process 16. The 
resulting table of test cases would be stored in database 22. 
The table of test cases could then be used as input to a filter 
process 24 (many are known in the art) to translate the table of 
test cases into a form readable by test script generation 

20 process 26 (also known in the art) to generate executable test 
scripts for a software system. 

The system described above implements our inventive 
methods for enumerating a minimal number of test cases. Our 
inventive methods are comprised of two general sets of 

25 procedures; one for generating a table of test cases for each 
individual relation as embodied in process 14 and another for 
combining the individual tables into a single table as embodied 
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in process 20. The building blocks used in each set of 
procedures are the fields, field values, relations, and degree 
of field interactions entered by the user of our inventive 
system. Our system and method can also be used to generate test 
cases for testing protocol conformance and feature interactions 
of software. In screen based database applications, fields 
correspond to the fields on the screen. In protocol conformance 
applications fields correspond to the decision points that 
determine a test scenario, such as choice of parameters and 
orderings of events. 

To create a relation, the user specifies the fields in the 
relation, a set of valid and invalid values for each field and 
constraints. The fields specified could all come from one 
screen or span a plurality of screens. Within screen 28 are 
examples of three fields (DEST 30, MODE 31, and PRI 32) . Table 3 
below depicts a relation of valid values for those fields (color 
or blank for DEST, 600dpi or blank for MODE, and D, D+ , or D- 
for PRI) related to the printing the output of a database query. 
These field values together define a relation. DEST represents 
destination printer, MODE represents mode, and PRI represents 
Table 3: Valid Values for relation 1 
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Table 4: Valid Values for relation 2 
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myprinter 


blank 
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blank 













the priority. In essence relation 1 says that when the priority 
is one of the deferred values (D,D-,D+), the valid values for 
DEST are color or blank and the valid values for MODE are 600 
dpi or blank. Table 4 depicts another relation for the fields 

10 DEST, MODE, and PRI. Table 4 indicates that when the valid field 
for PRI is I (immediate) the valid values for field DEST is 
myprinter or blank, and the valid value for MODE is blank. These 
two relations together check that the software that interfaces 
between a user and a printer enforces the requirements that the 

-5 local printer can be used only when the priority is immediate; 
and the color printer and 600 dpi mode can only be used when the 
priority is deferred. 

Once the user specifies the relationships among fields 
and values, the user also specifies the degree of interaction 

20 between fields that the user wishes to test; the interactions 
can be specified as any n-combination (e.g. pair-wise, triples, 
exhaustive, etc.). As an example, pair wise combinations mean 
that for any two fields f x and f 2 and any valid values, v 1 for f 1 
and v 2 for f 2 , there is some test case in which f^ has the value 

25 vj and f 2 has the value v 2 . Using relation 1 as an example, 

testing the pair-wise combinations of the valid values results 
in the test cases shown 61 in Figure 6. Exhaustive testing would 
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have required twelve test cases. For relation 2 there are two 
test cases 62 shown in Figure 6 for exhaustive or pairwise 
testing, alike. Thus, exhaustive testing for both relations 
would have required 14 test cases, while pair-wise testing 
requires a total of 8 test cases. 

The six test cases 61 depicted in Figure 6 for relation 1 
were generated using our procedures for generating a table of 
test cases as implemented in test case generation process 14. 
This process uses a new set of deterministic procedures based on 
known deterministic algorithms in combination with a random 
procedure based on a random algorithm we developed. In our 
embodiment of process 14, deterministic procedures are used 
first to attempt to produce a table. If deterministic 
procedures cannot directly be used because the number of fields 
and interactions are not conducive to a deterministic approach, 
then the relation is decomposed into sub-parts for which test 
cases can be generated using deterministic procedures. Process 
14 uses the generated test cases for the sub-parts as seed test 
cases for our random procedure which then completes the 
generation of a table of test cases for that relation. 
Determin e at. ic Procedures 

The first step in our deterministic process is to examine 
a relation and determine the group of fields with the largest 
number of field values or the largest group of fields with the 
same number of field values. The next step is then to examine 
this group of fields to see if a table of test cases can be 
generated using a deterministic algorithm; in our embodiment, 
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the deterministic algorithms are either our projective plane 
procedures, our application of the general product rule 
procedures, or just an exhaustive enumeration of all 
combinations of field values. If there is only one field with 
5 the largest number of field values, our deterministic 

procedures simply generate all n-way combinations (where n is 
the degree of interaction) for selected subset of fields with 
the highest number of field values using the well known in the 
art techniques of exhaustive enumeration. These test cases are 
10 then compared with test cases generated from our projective 

plane or product rules procedures from a subset of the relation 
with the largest number of fields having common values. This 
subset of fields is used to seed the random algorithm which is 
used for completing the table by filling in values for the 
15 remaining fields. 

Our generalized projective plane procedures are used when 
the number of values is a fixed prime power q and the number of 
fields is q+1. The classical projective plane construction as 
taught in the art is based on the geometry of the projective 
20 plane (see Hall Jr., M . , Combinatorial Theory, Wiley 
Interscience, New York, 1986) . It has been used for 
experimental design for pair-wise interactions. The generalized 
projective plane construction gives a table of <f test cases 
where g is the number of field values and n is the degree of 
25 interaction and n£q + 1 . Our generalized projective plane 
procedures develop n-way coverage as follows. 

The number of test cases necessary to cover n-way 
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interactions of q number of field values is cf 1 . Our procedure 
then lists the number of n-way combinations as (a lf . . . ,0^) with 
(0,...,0) being the first and (n-1 n-1) being the last. 
Our process then determines test cases of the form (a 1 ,...af), 
5 with f being the number of fields. For X from 0 to q, the value 

n 

^O^xX 1 * (mod q) is the value at field X for test case labeled 
i = i 

(a 1( ...,a„) . If f = q+l, then the value for a f is c^. The field 
q+1 is called the "line at infinity". For the situation when q 
is a prime power but not a prime number, an identification needs 
10 to be made between numbers 1 to q and the finite field with q 
elements . 

As an example of applying our procedures to a relation, 
assume the relation 30 as shown in Figure 3 with four fields 
each with three values. Further assume that the user wishes to 

15 generate test cases for pair-wise combinations of the fields. 
Our process numbers the test cases according to the number of 
pair wise combinations that can be generated as depicted in 
column 35. With three levels of values for each field, the 
number of pair wise combinations (n=2) that can be generated is 

20 nine (3 2 ) . Therefore, to generate the first of nine test cases 
of the form (a 1 ,a 2 ,a 3 , a 4 ) for pair-wise interactions, we use the 

n 

formula ^c^xX 1 " 1 (mod 3), where a x is the first value of the 
first pair 36 and X the number representing the field position 
37 from the vector (1, . . . , q) representing the number of fields. 
25 Therefore, the value for a x is a x times X 1 " 1 plus ot 2 times X 2 " 1 at 
field 1 (X=0) , which equals 0 shown at 38. The value for a 2 is 
a x times X 1 " 1 plus a 2 times X 2 " 1 at field 2 (X=l) which is also 0 
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shown at 39. This process is repeated for each a L for each of 
the nine test cases. The resultant table is a table of nine test 
cases 34 as shown in Figure 3. At first glance, it may not be 
apparent how the table 34 relates to the relation 30. In the 
table 34 each of the columns 33 represent the fields, and each 
number in the each cell within each column represents a field 
value. For example, looking at the fourth test case, labeled 
(0,1) 40, the 0 under the column a x indicates the first field 
value for field 1 and the 2 under column a 3 indicates the third 
value for field 3. Accordingly, the fourth test case shown in 
the table 34 for relation 30 is (v x , v 2 ' , v 3 ' ' , v 2 ' ' ' ) . 

The projective plane algorithm is only directly 
applicable when the number of fields values is q (a prime or 
power of prime) and the number of fields is less than or equal 
to q+1. (It is also effective when the number of values is close 
to a prime power, for example 10 is close to 11) . Often a 
relation does not easily fit the projective plane construct. 
Under those circumstances, our test case enumeration process 
breaks the relation into sub-relations to determine if test 
cases for a sub-relation can be generated deterministically , 
either using the projective plane procedures or, if it is small 
enough, through exhaustive enumeration. If so, the sub- 
relations are then combined using product rules. The product 
rules are only directly applicable whenever each field has the 
same number of field values and the degree of field interaction 
is pair-wise. 

As an example, consider a relation 43 shown in Figure 4 
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having 9 fields each with two field values. Our process would 
take a subset of the relation 44, for which our projective plane 
algorithm can be applied, and generate test cases for that 
subset. In this example, four test cases 45 were generated for 
relation 44 for pair wise coverage of three fields each with two 
field values. Our product rules produce a table of 8 test cases 
for nine fields 46. These test cases 46 are the product of the 
four test cases 45 combined with itself. The process for doing 
so is to repeat each of the elements in the first table by the 
number of fields in the second table and then repeat each of the 
sequence of elements for the fields in the second table by the 
number of fields in the first table. Our process then examines 
the resulting table of test cases looking for duplicates and 
removes any redundant cases. In our example above, the fifth 
test case 42 is a duplicate of the first test case 41 and 
therefore would be removed. 

Our implementation of the product rules can be 
generalizes as follows. Let S and T be two sets of test cases 
with the same number of values for each field. The number of 
fields may vary; as an example let S have m fields and T have r 
fields. A typical test case in S would be written as (s^ 
s m ) . A typical test case for T would be written as (t 1# . . .,t r ) . 
The product of the test cases for m*r fields would be of the 

form ( Sl s x ,s 2 s 2 s m sj repeated r times for each 

Si and (t x t r t! t r ,) where each test case 

(t x t r ) is repeated m times. 

An alternative approach called the reduced product rule 
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is applicable to the situation where there exists a test set T 
generated by the projective plane approach and a test set S 
having the same number of field values as T but an arbitrary 
number of fields . The reduced product rule approach reduces T by 
5 removing the values for the last field from the table of test 
cases and also removing the test cases in which all of the 
entries are the same. We then use our product rules to combine 
T' and S. To the result we add a column with the value 0 for the 
test cases from S and the value of the elements removed from T 

10 when the column was deleted for each of the test cases T' . 

As an example, let T be the table 50 as depicted in Figure 
5 which is a table of test cases generated by our projective 
plane algorithm for a relation having four fields and three 
field values. If our objective is to produce a table of test 

15 cases for 16 fields, each with three field values and pair wise 
interactions, our process would use the product rules to create 
a table which would have 17 test cases for 16 fields. Using our 
reduced product rule procedures would produce table 51 shown in 
Figure 5 having fifteen test cases 53 for thirteen fields 54. 

20 Such a table of test cases is created by modifying table 50 by 
removing the last column 55 and also removing the first three 
rows 56 because each of the values remaining are the same. The 
modified table 50 is then combined with the original table 50 
using our product rules to produce the table of fifteen test 

25 cases 51. 

RANDOM PPOnKGG 

Our random procedure is used to complete the generation 
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of a table of test cases when the deterministic procedures 
cannot be used to generate a complete table of test cases for a 
relation. For each new test case, our random procedure begins by 
generating a set of candidate test cases using our locally 
greedy algorithm described below. Given this set of candidate 
test cases, the algorithm picks the next test case using an 
evaluation function which assigns a score to each candidate 
test case. The test case added to the table is the candidate 
test case with the highest score. The two basic evaluation 
functions are the tt edge-greedy" evaluation function and the 
"balanced greedy" evaluation function. 

The basic steps that comprise our locally greedy random 
method are as follows: 

1. Select at random a (n-1) set (n = the degree of 
interaction) that has the highest priority. For 
example, in the case of pairwise coverage, this would 
be a (field, value) that belongs to the greatest 
number of uncovered pairs. 

2. Select at random an order for the remaining fields. 
Working in that order, for each new field, find the 
field values that give the highest score with respect 
to the evaluation function selected by the user for 
the run. Choose one of them at random and proceed to 
the next field. 

For example, using relation 1, if the values color and 
600 dpi have been found for fields DEST and MODE, 
respectfully, then for each value for field PRI, one 
of the evaluation functions is used to find the score 
for a test set including DEST with value color, MODE 
with value 600 dpi, and PRI with value D. 

3. After a value has been chosen for each factor, check 
that the generated test case meets all of the 
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constraints. If the resulting test case violates a 
constraint, then permute the field labels as they are 
associated with the specified values and see of the 
resultant test case violates the constraint. If this 
does not work not, discard the test case and generate 
another. If none of the generated test cases satisfies 
the constraints, the algorithm uses an exhaustive 
5 search to find a test case. If a constraint does not 

rule out a large percentage of the test cases, one of 
the generated candidate test cases is likely to 
satisfy it. The other constraints can be satisfied by 
having the evaluation function assign low scores to 
partial test cases that violate them. 

Evaluating the sample test cases according to our edge- 

0 greedy algorithm completes a local optimization of the number 
of new n-sets covered by each test case. Namely, the score 
assigned to a partial test case is the number of n-sets 
contained in the partial test cases that have not yet been 
covered previously. For example, for pair wise coverage, the 

5 score for a test case { (DEST, color) , (MODE, 600dpi), (PRI,D)} 
will be 2 if the previously selected test cases did not cover 
the two pairs {(DEST, color) , (MODE, 600dpi)}, { (MODE, 600dpi) 
covered by this test case. 

Another evaluation function is our balanced greedy 

0 algorithm. For pair-wise coverage the balanced greedy algorithm 
uses a steepest descent for a potential function on the space of 
graphs. Consider a graph with a vertex at (f,l) where f is a 
field and 1 a value. The graph has an edge between (f,l) and 
(f',1') if the pair (f,l), (f',1') isn't covered. Then each 

5 vertex (f,l) is assigned a tuple (n 1( ....,1^) where v is the 
number of fields and n i is the number of uncovered pairs { (f ,1) , 
(i,l')> for all possible choices. The potential function is the 
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sum or squares of the entries in the tuple. 

Another aspect of our random method is that it also 
incorporates cyclical block methods for generating a block of 
test cases. A cyclic block of test cases (mod 2) headed by a 
5 test case (ai,...,a f ) is the set of test cases (a^i, ...,a f +i) 
mod J., for 1< i <1, (for an arbitrary group G a G-block in the 

set of test cases (a x g ,a f g) for g in G) . In this block 

mode, a randomly generated test case is used as the header case 
and then a block of test cases is generated using the cyclical 

10 methods. When operating in the block mode, our process would use 
the cyclical procedures a limited number of times depending on 
the number of fields. The limit would by a user definable system 
parameter. If a set of test cases has a block structure, then an 
additional filed may be added without increasing the number of 

15 test cases. The additional field would be assigned a different 
value for each block of test cases, similar to the "line at 
infinity" shown in table 50. 

The user then can also specify constraints which are used 
by our method as unallowed test cases. A set of unallowed test 

20 cases can be specified using wild cards, *, or explicit values. 
Other constraints remove test cases from valid relations. For 
example, if a user specifies a test case that should not happen, 
that test case is removed from the test set. 

Once the appropriate number of test cases for the 

25 specified interactions for each relation is generated, our 

inventive method requires a user to identify fields that are 
non-interacting; i.e. fields not involved in any cross- 



- 18 - 



WO 96/11438 PCT/US95/14283 

validation rule. As an example, Table 63 shown in Figure 6 
depicts two fields that are not interacting. WC stands for a 
wire center with value WC38 and Head can have two values TOTAL 1 
and T0TAL2. 

5 aw sample appt.tcatton nv wnF! trst CASE GENEFPiTTON PROCEDURES 
Consider a relation 81 depicted in Figure 8 having 
fifteen fields with some of the fields having three field values 
82 and some of the fields have 2 field values 83. Our process 14 
determines the number of fields with the most number of field 
10 values. In this example, there are 10 fields with three field 
values. Because the fields have three field values and the 
number three is a prime number, our process 14 looks to apply 
projective plane procedures to generate a table of test cases 
for the ten fields 82. Our process 14 generates test cases using 
15 the projective plane procedures for four fields with three 

values which results in the table of test cases as depicted in 
table 50 in Figure 5. The process then applies our product rules 
to the table of test cases 50. Our process recognizes that 
applying the reduced product rule covers thirteen fields; 
20 whereas in this example it is only necessary to cover ten 
fields. However, process 14 uses the extra three fields to 
enumerate three of the fields having two field values 86. The 
entries into the table are evaluated according to the function 
MOD[2] and changed. This is depicted in the three columns 86 
25 shown in Figure 8. If you compare these three columns 86 to the 
three columns the table of test cases 56 in table 51, this 
change in readily apparent. Table 81 is then used to seed the 
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random algorithm, which then completes the table by filling in 
the remaining two columns 87 . 

TEST CASE MERGING PROCEDURES 

Another aspect of our inventive method is the process by 
5 which our inventive test script merging process 20 operates on 
all the tables generated by process 14. Using the relations 
defined above, our test case merging process 20 would combine 
the individual table of test case generated for each relation in 
accordance with the rules that follow: 
10 1. If two or more relations have fields in common, the 

combined test set for them is the union of the test 
sets generated for the individual relations; and 
2. For those relations that do not have fields in common, 
the combined test set is created by folding the 
15 smaller test set into the larger. 

Applying these rules to the test cased generated for relations 1 
(table 61), 2 (table 62) and 3 (table 63) which are shown in 
Figure 6 works as follows. Relation 1 and relation 2 have fields 
in common and therefore are combined to produce the union of 
20 test cases shown in table 71 shown in Figure 7. Using our second 
rule for combining relation 3 63 with the test cases 71 produces 
a consolidated table of test eight test cases 72. 

ADDITIONAL CAPABILITIES 

Our invention also provides the user the capability to 

25 ... 

test the reaction of the system under test to invalid inputs by 

specifying one or more invalid values for a field that occurs in 
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a relation. It is important to remember that the value is only 
invalid in the context of a relation and may be valid elsewhere. 
In the screen testing application, a test case containing an 
invalid value should produce the appropriate error message. Our 
5 system only designates one invalid value per test case to insure 
that the cause of the error message is identifiable. A system's 
reaction to n-way faults can be tested by making each faulty 
entry/ input a valid value in an n-way relation. System 
reactions to multiple faults can be tested by using relations 
10 with n-way interactions, with n being the number of faults. 

The user can also specify seed test cases for each 
relation. Seeded test cases can be either required or 
unallowable. Required seed cases will be guaranteed to be in the 
final set, while unallowable test cases are guaranteed not to 
15 appear in the final test set. Unallowed seed test cases are 

treated as constraints, while the required seed test cases are 
used to seed the random greedy algorithm. Since some of the 
interactions are already covered in the seed required cases, 
the system produces fewer additional test cases. 
20 The user can also define other constraints that are 

either required combinations or unallowed combinations where 
the relationship between the fields are unique combinations of 
fields or a unique order of values (e.g. increasing or 
decreasing) among fields. These constraints are used by our 
25 system and method to seed the random algorithms. 

Another aspect of our inventive method is to define a 
group of fields as a virtual field (hereinafter called 
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compounds) and use these compounds with other fields in 
defining a relation to be processed by our method. In 
processing, a compound is treated like a singular field. These 
relations including compounds would be stored in database 18 
5 and used by the system as a singular field to be built into 
relations and treated as a single field for our inventive 
process. 

Another aspect of our system is its application in screen 
based software systems to define relations as requirements for 

10 the software development. These relations can then by saved and 
used to test the resulting developed software to assure that the 
system operates in accordance with the defined requirements. 

Finally, although the system and method described herein 
is described in terms of its application to software, our 

15 invention is easily applicable to any system or process (i.e. 
manufacturing, hardware interaction, etc.) where there are 
multiple components or elements that must interact and it is 
impossible to test all interactions. Each of the components are 
analogous to the fields described above and each of the 

20 components has characteristics or features that are analogous 
to our field values. Our system and method can be used to 
generate tables test cases enumerating combinations of those 
components that interact. It therefore understood that the 
system and method for providing automated test case generation 

25 is not limited to the specific forms disclosed and illustrated, 
but may assume other embodiments limited only by the scope of 
the appended claims . 
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We claim: 

1. A method for generating a minimal plurality of test 
cases for testing software systems with the software having a 
plurality of data fields with multiple possible values, said 
method comprising the steps of: 

inputting into a data base all fields and field 

values ; 

defining a plurality of tables containing related 

field values; 

generating a table of test cases for each of said 
tables of field values using a user specified value for degree 
of interaction between fields; 

defining a plurality of tables of field values that 

are non- interacting; 

merging said tables of test cases into a single table 

of test cases; and 

merging said table of non-interacting fields with 

said single table of test cases. 

2. The method according to claim 1 wherein said step of 
merging said tables of test cases comprises; 

creating a union of said tables if two or more of said 
tables containing related fields have fields in common; and 

folding the table of test script into the table of 
largest test scripts whenever there are not any fields in 
common . 

3. The method according to claim 2 wherein said step of 
generating a table of test cases comprises: 

using a deterministic algorithm to attempt to 
generate said table; and 
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using a random algorithm for generating said table if 
said deterministic algorithm cannot produce a solution. 

4. The method according to claim 3 wherein said 
deterministic algorithm is a projective plane algorithm when 
the highest number of field values is a prime power and the 
number of fields is less than one more than said prime power. 

5. The method according to claim 4 wherein said 
deterministic algorithm is a product rule algorithm using a 
known construction as a seed test case. 

6. The method according to claim 5 wherein said random 
algorithm comprises: 

selecting at random a test case that belongs to the 
greatest number of uncovered combinations; 

selecting at random on order for the remaining 

fields ; 

for each new field selected find the field values that 
give the highest score for an evaluation function; and 

check that the generated test case meets all user 
defined constraints . 

7 . The method according to claim 5 wherein said 
evaluation function locally optimizes the number of new d-sets 
selected. 

8. The method according to claim 5 wherein said 
evaluation function is one that uses the steepest decent 
function on a plane of graphs. 

9. The method in accordance with claim 1 wherein the data 
is inputted through the screen and a database of the 
relationships is created. 

10. The method of claim 1 whereby the data is entered by 
the computer system capturing the screens generated by the 
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users and from this screen capture function a database of 
relationships is created. 

11. A system for the automatic generation of a minimum 
plurality of test cases for testing software systems; said 

system comprising: 

a display terminal for inputting values for fields on 
the screen and defining a plurality of relations among the 
fields; 

a database for storing said plurality of relations; 

a computer means for generating a set of test cases 
for each of said relations based on a user specified value for 
the number of interacting fields within the relations; 

a second computer means for combining said test cases 
for each of said plurality of relations and for said relations 
of non- interacting values; and 

output means for writing to a database the merged 
table which is the enumeration of all test cases. 

12. The system of claim 10 further comprising a test 
script generation means which accepts said table of test cases 
from said output means to generate a test script capable of 
being executed to automatically test software. 

13. The system of claim 10 wherein said system further 
comprises a capture means to capture user activities on a screen 
and create a database of fields, values and relations. 

14. The system of claim 10 wherein said computer means 
for generating a set of test cases is comprised of: 

a deterministic means for computing tables of test 
cases from relations; and 

a random means for generating tables of test cases 
from relations when said deterministic means is inapplicable. 
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15. A method for automatically generating a table of test 
cases for testing software systems which consists of and 
operate on data elements and values for data elements, said 
method executed by a computer comprising the steps of: 

identifying relationships between data elements; 
determining a table of test cases for each of said 
identified relationships based on user specified n-tuple 
combinations between values of said relationships said method 
of determining comprising; 

using deterministic procedures for determining 
said table of test cases; and 

using random procedures for determining said 
table of test cases whenever said deterministic procedures 
cannot produce a solution; and 

merging said tables of test cases into a single table 
by creating a union of the table of test cases whenever said 
tables have two or more data elements in common and if test 
cases do not have fields in common the smaller table is folded 
in the larger table. 

16. The method according to claim 14 wherein said 
deterministic procedures incorporate a projective plain 
algorithm. 

17. The method according to claim 15 wherein said random 
procedures further comprises the steps of: 

selecting at random a test case that belongs to the 
greatest number of uncovered combinations; 

selecting at random on order for the remaining data 

elements ; 

for each new data element selected find the data 
element values that give the highest score for an evaluation 
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function; and 

check that the generated test case meets all user 

defined constraints. 

18. The method according to claim 16 wherein said 
evaluation function determines a local optimal value, 

19 . The method according to claim 16 wherein said 
evaluation function determines the value based on the steepest 
descent for a potential function of a space of graphs. 

20. A method for automatically generating a table of test 
cases for testing combinations of components of a process or 
components of a system that interact, with each of said 
components having a plurality of characteristics, said method 
executed by a computer comprising the steps of: 

identifying relationships between components; 
determining a table of test cases of characteristic 
values for each of said identified relationships based on user 
specified n-tuple combinations of said components within said 
relationships, said method of determining comprising; 

using deterministic procedures for determining 
said table of test cases; and 

using random procedures for determining said 
table of test cases whenever said deterministic procedures 
cannot produce a solution; and 

merging said tables of test cases into a single table 
by creating a union of the table of test cases whenever said 
tables have two or more data elements in common and if test 
cases do not have fields in common the smaller table is folded 
in the larger table. 

21. The method according to claim 20 wherein said 
deterministic procedures incorporate a projective plain 
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algorithm. 

22. The method according to claim 21 wherein said random 
procedures further comprises the steps of: 

selecting at random a test case that belongs to the 
greatest number of uncovered combinations; 

selecting at random an order for the remaining 
components ; 

for each new component selected find the value of the 
characteristic for that component that gives the highest score 
for an evaluation function; and 

check that the generated test case meets all user 
defined constraints. 

23. The method according to claim 22 wherein said 
evaluation function determines a local optimal value. 

24. The method according to claim 22 wherein said 
evaluation function determines the value based on the steepest 
descent for a potential function of a space of graphs. 
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